{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0b_VKjjU_fsz"
   },
   "source": [
    "# RL environments\n",
    "\n",
    "In this book we will be using [gymnasium](https://gymnasium.farama.org/) for creating the RL environments. Gymnasium is a standard API for reinforcement learning, and a diverse collection of reference environments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q9KA7aE9_fs1"
   },
   "source": [
    "#### Running in Colab/Kaggle\n",
    "\n",
    "If you are running this on Colab or Kaggle, please uncomment below cell and run this to install required dependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install gymnasium[classic_control]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PilthBN9_fs3"
   },
   "source": [
    "#### Importing required packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "waewx-E5ysVn"
   },
   "outputs": [],
   "source": [
    "import gymnasium as gym\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JMdc791E_fs4"
   },
   "source": [
    "#### First Environment - Mountain Car\n",
    "\n",
    "The [Mountain Car MDP](https://gymnasium.farama.org/environments/classic_control/mountain_car/) is a deterministic MDP that consists of a car placed stochastically at the bottom of a sinusoidal valley, with the only possible actions being the accelerations that can be applied to the car in either direction. The goal of the MDP is to strategically accelerate the car to reach the goal state on top of the right hill. There are two versions of the mountain car domain in gymnasium: one with discrete actions and one with continuous. This version is the one with discrete actions.\n",
    "\n",
    "Each environment has to specify an `action space` which are the ways an agent/policy can interact with the environment. In case of Mountain car we have following possible actions:\n",
    "\n",
    "There are 3 discrete deterministic actions:\n",
    "\n",
    "0: Accelerate to the left<br/>\n",
    "1: Don’t accelerate<br/>\n",
    "2: Accelerate to the right<br/>\n",
    "\n",
    "Each environment's state is defined as `Observation Space` - i.e. the part of the state this is visible to the agent to make decisions.\n",
    "\n",
    "The observation for Mountain Car is a ndarray with shape (2,) where the elements correspond to the following:\n",
    "\n",
    "| Num | Observation                          | Min   | Max  | Unit         |\n",
    "|-----|--------------------------------------|-------|------|--------------|\n",
    "| 0   | position of the car along the x-axis | -1.2  | 0.6  | position (m) |\n",
    "| 1   | velocity of the car                  | -0.07 | 0.07 | velocity (v) |\n",
    "\n",
    "\n",
    "We use an agent/policy to take an action using the following method in the `environment`<br/>\n",
    "`action = policy(observation)`\n",
    "\n",
    "Then we pass the action to the environment to act using `env.step(action)`. The function call returns a tuple of five values: `observation, reward, terminated, truncated, info`. the returned\n",
    "\n",
    "- `observation` is the new state environment has moved into as a result of our action.\n",
    "- `reward` is the numerical reward our agent gets from environment as a result of the action it took.\n",
    "- `terminated` is a boolean which is true when the agent's action has resulted in completion of the task or agent has moved into a bad state (like falling off the cliff) from which it cannot recover. In other words, good or bad, the agent's action has resulted into a new state which is a terminal state from which agent cannot recover.\n",
    "- `truncated` is a boolean which is true when the agent could not complete the task and timeout happens. There are a few other scenarios when this could happen as a result of other actions. You can read more about it in the documentation for `gymnasium`.\n",
    "- `info` contains certain other information for debugging the environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 691
    },
    "id": "gLOKb2WrycEf",
    "outputId": "3def88f6-691d-4489-fbee-916fc6773ffa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "initial observation: [-0.5901667  0.       ]\n",
      "possible actions: 3\n",
      "\n",
      "taking action: 0\n",
      "got reward: -1.0. New state/observation is: [-5.9067070e-01 -5.0405273e-04]\n",
      "\n",
      "taking action: 0\n",
      "got reward: -1.0. New state/observation is: [-0.5916751 -0.0010044]\n",
      "\n",
      "taking action: 2\n",
      "got reward: -1.0. New state/observation is: [-5.911725e-01  5.026300e-04]\n",
      "\n",
      "taking action: 2\n",
      "got reward: -1.0. New state/observation is: [-0.5891665   0.00200597]\n",
      "\n",
      "taking action: 1\n",
      "got reward: -1.0. New state/observation is: [-0.58667195  0.00249456]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAF7CAYAAAD4/3BBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/SrBM8AAAACXBIWXMAAA9hAAAPYQGoP6dpAABNwElEQVR4nO3de1xUdeI+8GdmYIabM4hyVUBUFEHwgojjFYXEC97dvGtqurlomX3VKMvcbbNyK7uJVpta6qpZmlJeEBUz8S6pGKSmQnFVZAYQGJj5/P5ond+SWqLAmYHn/Xqd18qcMzPPnEXn6ZzP+RyZEEKAiIiIyILIpQ5ARERE9HssKERERGRxWFCIiIjI4rCgEBERkcVhQSEiIiKLw4JCREREFocFhYiIiCwOCwoRERFZHBYUIiIisjgsKERERGRxJC0oH374IVq1agU7OzuEh4fjxIkTUsYhIiIiCyFZQdmyZQsWLFiApUuX4syZM+jUqROio6ORn58vVSQiIiKyEDKpbhYYHh6OsLAwfPDBBwAAk8kEb29vzJs3D88//7wUkYiIiMhC2EjxpgaDAadPn0ZcXJz5MblcjqioKKSkpNy1fUVFBSoqKsw/m0wmFBYWolmzZpDJZPWSmYiIiB6NEALFxcXw8vKCXP7HJ3EkKSg3btyA0WiEu7t7tcfd3d2Rnp5+1/bLly/HsmXL6iseERER1aGsrCy0bNnyD7eRpKDUVFxcHBYsWGD+WafTwcfHB1lZWVCr1RImIyIiogel1+vh7e2NJk2a/Om2khSU5s2bQ6FQIC8vr9rjeXl58PDwuGt7lUoFlUp11+NqtZoFhYiIyMo8yPAMSa7iUSqVCA0NRVJSkvkxk8mEpKQkaLVaKSIRERGRBZHsFM+CBQswbdo0dOvWDd27d8fKlStRWlqK6dOnSxWJiIiILIRkBWXcuHEoKCjAyy+/jNzcXHTu3Bl79uy5a+AsERERNT6SzYPyKPR6PTQaDXQ6HcegEBERWYmafH/zXjxERERkcVhQiIiIyOKwoBAREZHFYUEhIiIii8OCQkRERBbHKqa6JyIiotp3vwt5LeFGvCwoREREjZTReBNpaR3h6NgdDg7d4egYBgeHrpDJlJDJbCCT2f53qf/CwoJCRETUSAkhUFWVB51uF3S6Xf991Ab29iGwtw+Bg0MI7O2DYWPTDAqFxrzIZHVfH1hQiIiI6H9UoazsDMrKzqCw8LdHlMpWUCpbQ6VqDZXKD0qlN2xtvaFUtoBS2QJyuUOtp2BBISIioj9kMFyDwXANJSUHAAAKRVPY2LjBxsYVtrauUCpbw86uA+zsAmBvHwAbm2aP/J4sKERERFQjRuMtGI23UFGRAQCQyZSQyx0hlztALndEy5b/grPzsEd6DxYUIiIiqpHfBs6qIJOpIJeroFL5w9ExHA4OYXB0DINS2fKR34MFhYiIiP6QQuEMhcIFNjZNoVC4wM6uPeztg2FvHwR7+45QKDS1/p4sKERERPQ/5FAqff5n8YZS6Qel0hcqVSsolb6Qy1V1noIFhYiIqBGTyexgbx8EO7sg2NsHws6uA2xsmkOhaAYbGxfY2DSDTFb/E8+zoBARETVSej3w6qudsWXLjv+OJ7H779gSziRLREREEjGZgJs3lbC1dZc6yl14s0AiIiKyOCwoREREZHFYUIiIiMjisKAQERGRxWFBISIiIovDgkJEREQWhwWFiIiILA4LChEREVkcFhQiIiKyOCwoREREZHFYUIiIiMjisKAQERGRxWFBISIiIotT6wXllVdegUwmq7YEBASY15eXlyM2NhbNmjWDk5MTxowZg7y8vNqOQURERFasTo6gBAUFIScnx7wcOXLEvO7ZZ5/Frl278MUXXyA5ORnZ2dkYPXp0XcQgIiIiK2VTJy9qYwMPD4+7HtfpdPj3v/+NTZs2YcCAAQCAtWvXokOHDjh27Bh69OhRF3GIiIjIytTJEZRLly7By8sLrVu3xqRJk5CZmQkAOH36NCorKxEVFWXeNiAgAD4+PkhJSbnv61VUVECv11dbiIiIqOGq9YISHh6OdevWYc+ePYiPj8fVq1fRp08fFBcXIzc3F0qlEs7OztWe4+7ujtzc3Pu+5vLly6HRaMyLt7d3bccmIiIiC1Lrp3gGDx5s/nNISAjCw8Ph6+uLrVu3wt7e/qFeMy4uDgsWLDD/rNfrWVKIiIgasDq/zNjZ2Rnt2rXD5cuX4eHhAYPBgKKiomrb5OXl3XPMyh0qlQpqtbraQkRERA1XnReUkpISXLlyBZ6enggNDYWtrS2SkpLM6zMyMpCZmQmtVlvXUYiIiMhK1Popnv/7v//DsGHD4Ovri+zsbCxduhQKhQITJkyARqPBzJkzsWDBAri4uECtVmPevHnQarW8goeIiIjMar2g/PLLL5gwYQJu3rwJV1dX9O7dG8eOHYOrqysA4J133oFcLseYMWNQUVGB6OhorFq1qrZjEBERkRWTCSGE1CFqSq/XQ6PRQKfTcTwKERHRQyooKMDYsWORnJxcL+9Xk+9v3ouHiIiILA4LChEREVkcFhQiIiKyOCwoREREZHHq5GaBREREZD2EEKisrER5eTkUCgWEEBBCwGg0wmAwwMHBAQqFAgqFAnK5HAqFAgAgk8nqLBMLChERUSNRVVWFmzdvIicnB7m5udDr9SgoKMCGDRug0+nw66+/wsPDA0IImEwmlJSUICsrC4GBgbC1tYWNjQ1kMhlUKhWcnZ3Ns7s7OTnB1tYWrVq1MpeXR8WCQkRE1ECVlpbizJkzSE1NhV6vR15eHlQqFQwGA4qLi+Hl5WW+BY1KpULbtm2hVqvNR0uEEGjTpg3s7e1hMBhQUVGB8vJy6PV63LhxA5WVlTAYDOZZ4/38/ODr64s2bdqY/+zo6PhQ2TkPChERUQMhhMCZM2dw4cIFHDt2DNeuXYOLiwvc3Nyg1WrRsmVLODk5wcHBAUqlEg4ODsjOzkZQUFCNjnwIIVBWVmZebt++jaKiIpSVleHatWu4du0aLl++jGvXrqFXr16IiIhAz549AeCBv79ZUIiIiKzQnXEjlZWVuHnzJnbt2oUdO3agqKgIgwcPRkREBIKDg2Fvbw+FQgFbW1soFIo6HTcihEBVVZV5KSsrw5EjR5CUlIQTJ07Az88PX3zxBQsKERFRQ2MymXDjxg1cu3YNBw4cwKVLl5CZmYnBgwcjJiYGbdq0gVz+/y/SrctC8iDu1AyDwYDvvvsOjz322AN9f3MMChERkRUoLi7GlStX8MMPP+CXX35BYWEhXF1d8dRTT6Fbt26SF5H7uZNLpVKhe/fuD/w8FhQiIiILdvv2bezbtw979uyBi4sLfH190bdvXwQFBcHFxUXqeHWGBYWIiMjC3Dktsnv3bmzYsAEuLi4YNmwYOnXqBDc3NyiVSokT1j0WFCIiIgtxZ+6RPXv24OOPP0ZAQABefPFFtG3bFra2ttXGljR0LChEREQWICcnB+fOnUNCQgKqqqqwcuVKdOjQoVGVkv/FgkJERCSh7OxsfPfdd7h06RKqqqowbdo0dOnSpdZmZLVWLChEREQSMBgM2LdvH3bs2IHAwEBERUWhS5cuUKlUUkezCCwoRERE9UgIgdLSUixbtgzFxcWYMmUKQkND4ejoaLGXCkuBBYWIiKgeVFVVQafTYf/+/Xj33Xfx/PPPY/DgweYb8FF1LChERER1rLKyEklJSfjiiy/QunVr7NmzhzOh/wkWFCIiojp0/fp1bNmyBRUVFXj88ccRERHBcSYPgAWFiIioDgghsGfPHuzbtw99+/ZFz5494e7uLnUsq8GCQkREVIuEEMjKysIrr7wCjUaD2NhY+Pn5NfrLhmuKBYWIiKiWVFZW4ueff8Zbb72FTp06ITY2FgqFgoNgHwILChERUS3Iz89HcnIyDhw4gCeffLJGd+6lu7GgEBERPaKMjAx8/fXXcHR0xD/+8Q80b95c6khWjwWFiIjoIZlMJiQlJWHz5s2YOHEievbsCXt7e6ljNQgsKERERA+hsrISGzZswPHjx/HGG2+gadOmHAhbi1hQiIiIasBkMiEvLw///ve/oVKpEB8fDwAcCFvLWFCIiIgeUHl5Ob7//nskJyejc+fOGDZsGItJHWFBISIiegAmkwlff/019uzZg7/97W/o0qULbGz4NVpX5DV9wuHDhzFs2DB4eXlBJpNhx44d1dYLIfDyyy/D09MT9vb2iIqKwqVLl6ptU1hYiEmTJkGtVsPZ2RkzZ85ESUnJI30QIiKiuvTBBx/g559/xpIlSxAWFsZyUsdqXFBKS0vRqVMnfPjhh/dc/+abb+K9997D6tWrcfz4cTg6OiI6Ohrl5eXmbSZNmoS0tDQkJiYiISEBhw8fxuzZsx/+UxAREdUBIQTKysrw0ksvQaVSYf78+WjTpo3UsRoFmRBCPPSTZTJs374dI0eOBPDb/5FeXl547rnn8H//938AAJ1OB3d3d6xbtw7jx4/Hjz/+iMDAQJw8eRLdunUDAOzZswdDhgzBL7/8Ai8vrz99X71eD41GA51Ox7tBEhFRnTAajfjpp5+wceNGBAcHY9SoUVAqlVLHsmo1+f6u8RGUP3L16lXk5uYiKirK/JhGo0F4eDhSUlIAACkpKXB2djaXEwCIioqCXC7H8ePH7/m6FRUV0Ov11RYiIqK6IoTAqVOnsGLFCvTp0wdjx45lOalntVpQcnNzAeCuuzW6u7ub1+Xm5sLNza3aehsbG7i4uJi3+b3ly5dDo9GYF29v79qMTUREVM2hQ4eQkJCAqVOnIjo6mvObSKBWC0pdiYuLg06nMy9ZWVlSRyIiogZICIHt27fj0KFDmDVrFiIiIqSO1GjV6hBkDw8PAEBeXh48PT3Nj+fl5aFz587mbfLz86s9r6qqCoWFhebn/55KpYJKparNqERERNVUVlZi165dyMjIwNy5c3k/HYnV6hEUPz8/eHh4ICkpyfyYXq/H8ePHodVqAQBarRZFRUU4ffq0eZsDBw7AZDIhPDy8NuMQERH9KSEEDAYDtm3bhsuXL+PJJ5+Eq6srJ2CTWI2PoJSUlODy5cvmn69evYrU1FS4uLjAx8cH8+fPx6uvvgp/f3/4+fnhpZdegpeXl/lKnw4dOmDQoEGYNWsWVq9ejcrKSsydOxfjx49/oCt4iIiIatuqVaug1+vx9NNPw9nZWeo4hIcoKKdOnUL//v3NPy9YsAAAMG3aNKxbtw6LFi1CaWkpZs+ejaKiIvTu3Rt79uyBnZ2d+TkbN27E3LlzERkZCblcjjFjxuC9996rhY9DRET04CoqKvDSSy+hS5cuePLJJ+Hk5CR1JPqvR5oHRSqcB4WIiB6FEAK3b9/GP//5T/Tt2xdRUVGcGbYe1OT7m/9vEBFRoyKEwK1bt7B27VqEhYVh4MCBkMut4qLWRoUFhYiIGpX8/Hx8+umnaNmyJUaNGiV1HLoPVkYiImo08vPzsXr1anh4eGDKlClSx6E/wCMoRETUKOTl5WHVqlXo169ftYs9yDKxoBARUYMmhMDNmzfx8ccfIzIyEr179+YcJ1aABYWIiBqsO+Vk06ZN6Ny5M/r06cNyYiVYUIiIqMG6du0aNm/ejNatWyMmJkbqOFQDHCRLREQNUmFhIVauXIkWLVpg3LhxUsehGuIRFCIianCKi4vxr3/9C8OHD8eAAQOkjkMPgQWFiIgaDCEEysvL8eGHH6J3796IiIjgmBMrxYJCREQNhsFgwMaNG9G8eXMMHjyY5cSKcQwKERE1CCaTCevXr0dRURFmzJjBcmLleASFiIgahLfffhsymQzz5s3jvXUaABYUIiKyevHx8WjSpAmmTJkClUoldRyqBSwoRERktYxGI3bu3Amj0YiJEyfC3t5e6khUS3gMjIiIrJLRaMT333+PK1euYPTo0dBoNBx30oCwoBARkdURQuDUqVM4cuQIhg8fDi8vL6kjUS1jQSEiIquTkJCAlStXYtSoUWjXrp3UcagOcAwKERFZDSEErl+/jm3btmHJkiXo0KGD1JGojvAIChERWQUhBAoKCvDee+9h0aJFCAwMlDoS1SEeQSEiIqtQXFyM9evXIzIyEkFBQVLHoTrGIyhERGTxDAYDNm3aBDc3N0RFRUkdh+oBj6AQEZHFW716NeRyOcaMGcOJ2BoJFhQiIrJYQgj885//RHp6OlavXg0nJyepI1E9YUEhIiKLZDQacfjwYZSVlSE+Pp7lpJHhGBQiIrI4JpMJaWlpOHz4MGbNmoUmTZpIHYnqGQsKERFZnPz8fHz55ZcYPHgwWrVqJXUckgALChERWRSDwYCVK1eiZ8+eCA0NlToOSYQFhYiILIbRaMSyZcvQuXNnDBgwAAqFQupIJBEWFCIisggVFRV44YUXkJOTg8cffxy2trZSRyIJ1bigHD58GMOGDYOXlxdkMhl27NhRbf0TTzwBmUxWbRk0aFC1bQoLCzFp0iSo1Wo4Oztj5syZKCkpeaQPQkRE1quyshL79++Hi4sL3n//fcjl/O/nxq7GvwGlpaXo1KkTPvzww/tuM2jQIOTk5JiX//znP9XWT5o0CWlpaUhMTERCQgIOHz6M2bNn1zw9ERE1COfPn8epU6cwadIkODo6Sh2HLECN50EZPHgwBg8e/IfbqFQqeHh43HPdjz/+iD179uDkyZPo1q0bAOD999/HkCFD8K9//QteXl41jURERFYsPz8fW7Zswfjx49GiRQup45CFqJNjaIcOHYKbmxvat2+POXPm4ObNm+Z1KSkpcHZ2NpcTAIiKioJcLsfx48fv+XoVFRXQ6/XVFgDYvHkzTCZTXXwEIiKqBxUVFVixYgUiIyMREhICmUwmdSSyELVeUAYNGoTPPvsMSUlJeOONN5CcnIzBgwfDaDQCAHJzc+Hm5lbtOTY2NnBxcUFubu49X3P58uXQaDTmxdvbGwBw/fp1HDlyxPzaRERkPXQ6Hd566y0EBgbiscce4xU7VE2tF5Tx48dj+PDhCA4OxsiRI5GQkICTJ0/i0KFDD/2acXFx0Ol05iUrKwsAMHLkSBw8eBA///wzhBC19AmIiKiuVVRU4JNPPkFRURGmTZvGIyd0lzofJt26dWs0b94cly9fBgB4eHggPz+/2jZVVVUoLCy877gVlUoFtVpdbQGA9u3bo1evXtiwYQOvAiIisiIHDhxAWVkZXnjhBV6xQ/dU578Vv/zyC27evAlPT08AgFarRVFREU6fPm3e5sCBAzCZTAgPD6/x6/fr1w+BgYFYuXIlj6IQEVmB9PR0nDhxAo8//jg0Go3UcchC1biglJSUIDU1FampqQCAq1evIjU1FZmZmSgpKcHChQtx7NgxXLt2DUlJSRgxYgTatm2L6OhoAECHDh0waNAgzJo1CydOnMD333+PuXPnYvz48Q91BY+trS3GjBkDg8GAd955B5WVlTV+DSIiqntCCNy8eRNbt25FREQE2rZty1M7dF81LiinTp1Cly5d0KVLFwDAggUL0KVLF7z88stQKBQ4d+4chg8fjnbt2mHmzJkIDQ3Fd999B5VKZX6NjRs3IiAgAJGRkRgyZAh69+6Njz766KE/hI2NDZYsWYLTp09j27ZtvLKHiMgCVVRUYP369fDx8UHfvn15aof+kExY4XkRvV4PjUYDnU5nHo8C/HY6adWqVRg/fjxCQkIkTEhERL+3evVqFBUV4fnnn5c6Cknkft/f99Kg6qunpyeGDh2Kb7/9Fjk5OVLHISKi/9qwYQNSU1Mxb948qaOQlWhQBUWhUCAsLAy+vr748ssvUVFRIXUkIqJGTQiBM2fO4NKlS3j22Wfh4OAgdSSyEg2qoACAUqnE448/jtzcXCQkJHA8ChGRRIQQyM3Nxd69ezFw4EC0a9eOg2LpgTW4ggL8diTl1VdfxX/+8x8cPXpU6jhERI1SZWUltm/fDg8PD/Tq1YvlhGqkQRaUO1577TV88sknOHPmjNRRiIgaFSEEdu7ciYKCAkyYMEHqOGSFGnRBadOmDZ544gl8++23+OWXX6SOQ0TUaBw4cACnT59GbGws7OzspI5DVqhBFxSFQoFevXrBz88Pu3fvRllZmdSRiIgaNCEETp48iQ8//BB/+9vf0Lx5c6kjkZVq0AUF+G2m2fHjx+PixYs4deoUp8MnIqpDhYWFWLduHV588UW0bNlS6jhkxRp8QQF+O5KycOFCbNmyBWlpaVLHISJqkG7fvo0dO3agZ8+e6NixIwfF0iNpFAUFALy8vDBr1ix8+umnuHbtmtRxiIgaFKPRiO+++w63bt1CdHR0tdubED2MRlNQACAkJAQjRozAsmXLUFBQIHUcIqIG49q1a/jyyy8xbtw4jjuhWtGoCopMJoNWq0Xfvn2xceNGTuJGRFQLqqqqMHv2bLz44ovw9vaWOg41EI2qoAC/DZqNiYlBZWUlkpOTYTQapY5ERGS1dDodFi1ahEWLFsHHx0fqONSANLqCIpPJ4OrqiujoaCQnJ+PatWu8soeI6CHcvn0bn376Kezs7NCnTx8OiqVa1egKyh0hISHo3bs33n33XZ7qISKqISEETp06haKiIjz99NO8CSDVukZbUACgf//+CAkJwYoVK6SOQkRkVfLz8/Htt99i1KhR8PDwkDoONUCNuqAoFApMmTIF5eXl2Lp1K8ejEBE9AIPBgPj4ePTo0QPBwcFSx6EGqlEXFABQKpX461//io0bNyIpKYnjUYiI/oDRaMSGDRugUqkwYsQIKBQKqSNRA9XoC4pMJoOnpydeeOEFHD16FPn5+VJHIiKyWIcOHcKpU6ewePFiDoqlOtXoC8odnTp1Qrt27bBz507eVJCI6B6OHDmCzz//HAsWLGA5oTrHgvJfdnZ2iImJQVZWFo4cOcJTPURE/yMnJwd79uzBmDFj0KpVKxYUqnM2UgewJGq1Gi+88AKio6PRqVMnuLm5SR2JiEhylZWVOHjwINzc3DB48GDY2PCrg+oej6D8jp2dHT766CMsWrSI41GIqNETQuDs2bM4fvw4pk+fznJC9YYF5R78/f0xdOhQ/Otf/0J2drbUcYiIJHPlyhVs3LgRc+bMQZMmTaSOQ40IC8o9yOVyREdHw8nJCYmJiZwfhYgaJZ1Oh7feegsTJ05EQECA1HGokWFBuQ+1Wo3p06cjIyMD6enpHDRLRI2KEAIrV65EZGQkwsLCpI5DjRALyh9o2bIlhg4dio0bN+LWrVssKUTUKBiNRnz22WfIyclBv379eMUOSYIF5Q/IZDL06tUL/v7++OSTT1BVVSV1JCKiOiWEQHp6OtLT0zFv3jy4urqyoJAkWFAewPTp01FYWIjt27dLHYWIqE6VlZVh27Zt6Nu3L4KCgqSOQ40YC8oDeu6555CamoojR45IHYWIqE4IIbB27Vq4uroiMjJS6jjUyNWooCxfvhxhYWFo0qQJ3NzcMHLkSGRkZFTbpry8HLGxsWjWrBmcnJwwZswY5OXlVdsmMzMTQ4cOhYODA9zc3LBw4UKLP33SvHlzDBs2DG+99RZ++OEHjkchogZFCIHExERcuXIF06ZNg1KplDoSNXI1KijJycmIjY3FsWPHkJiYiMrKSgwcOBClpaXmbZ599lns2rULX3zxBZKTk5GdnY3Ro0eb1xuNRgwdOhQGgwFHjx7F+vXrsW7dOrz88su196nqgEwmQ48ePTBu3DgkJyejvLxc6khERLXm559/xr///W/8/e9/h6Ojo9RxiCATj3AooKCgAG5ubkhOTkbfvn2h0+ng6uqKTZs2YezYsQCA9PR0dOjQASkpKejRowd2796NmJgYZGdnw93dHQCwevVqLF68GAUFBQ/U2vV6PTQaDXQ6HdRq9cPGfyh6vR4ffPABQkNDERUVxVuNE5HVy8nJwRtvvIFx48YhPDwccjnP/lPdqMn39yP9Fup0OgCAi4sLAOD06dOorKxEVFSUeZuAgAD4+PggJSUFAJCSkoLg4GBzOQGA6Oho6PV6pKWl3fN9KioqoNfrqy1SUavVGD9+PBITE+86vUVEZG2Ki4uxZcsWtGzZEkFBQSwnZDEe+jfRZDJh/vz56NWrFzp27AgAyM3NhVKphLOzc7Vt3d3dkZuba97mf8vJnfV31t3L8uXLodFozIu3t/fDxq4VrVu3xvTp07FkyRJUVFRImoWI6GGZTCZcuHAB2dnZmDZtWr0fkSb6Iw9dUGJjY3HhwgVs3ry5NvPcU1xcHHQ6nXnJysqq8/f8M4GBgZg2bRpefvllmEwmqeMQEdVYSUkJ3nvvPfz1r3+Fq6ur1HGIqnmogjJ37lwkJCTg4MGDaNmypflxDw8PGAwGFBUVVds+Ly8PHh4e5m1+f1XPnZ/vbPN7KpUKarW62iI1mUyGgQMHwsPDA5s2beKRFCKyKqWlpVi2bBmmT5+O1q1bSx2H6C41KihCCMydOxfbt2/HgQMH4OfnV219aGgobG1tkZSUZH4sIyMDmZmZ0Gq1AACtVovz588jPz/fvE1iYiLUajUCAwMf5bPUO3t7e8TExODw4cM4ffo0Lz0mIqtQWVmJdevWoXXr1njsscc4UyxZJJuabBwbG4tNmzbh66+/RpMmTcxjRjQaDezt7aHRaDBz5kwsWLAALi4uUKvVmDdvHrRaLXr06AEAGDhwIAIDAzFlyhS8+eabyM3NxZIlSxAbGwuVSlX7n7CO+fv7Y/z48Thw4AACAgLMA4aJiCxVUlIS8vPzsXjxYpYTslg1OoISHx8PnU6HiIgIeHp6mpctW7aYt3nnnXcQExODMWPGoG/fvvDw8MBXX31lXq9QKJCQkACFQgGtVovJkydj6tSp+Pvf/157n6qe9erVCx4eHti6dSuMRqPUcYiI7is1NRW7du3C6NGjYW9vL3Ucovt6pHlQpCLlPCj3U1FRgWeeeQYTJkxAv379pI5DRFSNEAKFhYV48803ERISgvHjx3MeJ6p39TYPCv1/KpUKH3zwAZYuXYrs7Gyp4xARVSOEwNGjR2Fra4uJEyeynJDFY0GpRQqFAm+88QZWrVp13zldiIikcObMGezduxfPPPMMx52QVWBBqUUymQydO3dG+/bt8dlnn5ln2iUiktL169exdu1axMbGcr4TshosKLVMpVJh6NChuHnzJs6ePSt1HCJq5IxGI1577TVMnjwZAQEBUschemAsKHWgadOmmDZtGnbu3Ilff/2V86MQkSSqqqqwfv16hIWFoUuXLjy1Q1aFBaUOyGQyBAYGIiIiAmvWrMHt27eljkREjYzRaERycjLS0tIQGRkJOzs7qSMR1QgLSh0aPnw4nJ2d8emnn0odhYgamcLCQmzduhUxMTF3zfpNZA1YUOrYU089hezsbOzdu1fqKETUSJhMJqxduxZarRYRERFSxyF6KCwodcze3h4zZ87Ed999h4yMDI5HIaI6JYTAtm3bUFZWhnHjxnHcCVktFpQ6JpPJ0KZNG/To0QPr16/HrVu3pI5ERA3YDz/8gF27dmHx4sWcyp6sGgtKPZDJZOjfvz/s7e1x4MABVFVVSR2JiBqg/Px8fPTRR3jppZes8uarRP+LBaWeODo6Yvr06Th79izOnj3LUz1EVKuKioqwdetWDBgwAL6+vjy1Q1aPBaUetWzZElOmTME777zDWWaJqNZUVlYiISEBOTk5iIyM5NETahBYUOpZQEAAZs2ahUWLFsFkMkkdh4isnBACt27dwq5duzBnzhw0bdpU6khEtYIFRQK9e/dG165dsW7dOlRWVkodh4isWGlpKZYuXYpnnnkGLVq0kDoOUa1hQZGAra0tRo0ahezsbBw/fpxHUojooZSXl2P16tXo3r07evbsyXEn1KCwoEjE3d0dERER+PLLL/Hrr79KHYeIrFBCQgLKy8sxefJkqaMQ1ToWFAmFhoaiU6dO2LZtGwwGg9RxiMiKnD17FhcvXsTkyZNhY2MjdRyiWseCIiF7e3uMHz8eBQUF2Lt3Ly89JqI/JYRAfn4+EhMT0bt3b/j4+PDUDjVILCgSs7Ozw6uvvor3338fV65ckToOEVm4yspKfP755ygrK0P//v0hl/OfcWqY+JttAWQyGd5991289957+OWXX6SOQ0QW7OTJk8jMzMSCBQt45IQaNBYUCyCTyeDv74+oqCh89dVXKCoqkjoSEVmgH3/8EZs3b8bTTz+NJk2aSB2HqE6xoFgIGxsbREZGoqqqCgcPHuSlx0RUTXFxMd5++23MmDEDbdq0kToOUZ1jQbEgjo6OiI6OxoYNG/Djjz9y0CwRAQCMRiPi4+MRHR2N4OBgqeMQ1QsWFAsTGBiI+fPnY/369bh165bUcYhIYlVVVUhKSoKNjQ369+8PhUIhdSSiesGCYmFkMhn69OmDkJAQfPDBB6iqqpI6EhFJRAiB9PR0HDp0CNHR0WjWrBkHxlKjwYJioSZNmgQA2Lx5s8RJiEgqlZWVWLNmDbp3746goCCp4xDVKxYUCzZ79mxcvnwZR48e5XgUokZGCIFVq1YhMDAQQ4YMkToOUb1jQbFQMpkM7u7uGDZsGPbv349ff/2VJYWokTCZTNi7dy+ysrIwc+ZMKJVKqSMR1TsWFAsmk8kQGhoKLy8vfPHFFygrK5M6EhHVg4yMDOzYsQMvvvgiywk1WiwoViAmJgaZmZnYv38/j6IQNXB5eXnYvn07Jk+eDGdnZ6njEEmmRgVl+fLlCAsLQ5MmTeDm5oaRI0ciIyOj2jYRERGQyWTVlqeeeqraNpmZmRg6dCgcHBzg5uaGhQsX8mqVP+Dh4YEFCxYgKSkJ586dkzoOEdWRsrIy7NmzBy1btkTXrl15nx1q1Gr025+cnIzY2FgcO3YMiYmJqKysxMCBA1FaWlptu1mzZiEnJ8e8vPnmm+Z1RqMRQ4cOhcFgwNGjR7F+/XqsW7cOL7/8cu18ogbK29sbzz33HJYsWYLi4mKp4xBRLRNC4MCBAzh8+DBGjhwJBwcHqSMRSUomHuGcQUFBAdzc3JCcnIy+ffsC+O0ISufOnbFy5cp7Pmf37t2IiYlBdnY23N3dAQCrV6/G4sWLUVBQ8EDnW/V6PTQaDXQ6HdRq9cPGtzpCCBw6dAjffvstXn31VahUKqkjEVEtKSwsxLhx47Bhwwbzv41EDU1Nvr8f6fihTqcDALi4uFR7fOPGjWjevDk6duyIuLg43L5927wuJSUFwcHB1f4CRkdHQ6/XIy0t7Z7vU1FRAb1eX21pjGQyGbp37w5/f3/s2LEDFRUVUkciolpQUFCAZcuW4R//+AfLCdF/PXRBMZlMmD9/Pnr16oWOHTuaH584cSI2bNiAgwcPIi4uDp9//jkmT55sXp+bm3vXX8A7P+fm5t7zvZYvXw6NRmNevL29Hza21XN0dMSQIUNw6dIlnD17ljcVJLJyJSUl+Pzzz9GzZ090795d6jhEFsPmYZ8YGxuLCxcu4MiRI9Uenz17tvnPwcHB8PT0RGRkJK5cufLQd+CMi4vDggULzD/r9fpGXVJatmyJ/v37Iz4+Hu3atbvrCBYRWY+EhATY2tpi2LBhHBRL9D8e6m/D3LlzkZCQgIMHD6Jly5Z/uG14eDgA4PLlywB+uyIlLy+v2jZ3fvbw8Ljna6hUKqjV6mpLY9e9e3cMHz4cK1as4FEUIiskhMD58+eRnp6OYcOGwd7eXupIRBalRgVFCIG5c+di+/btOHDgAPz8/P70OampqQAAT09PAIBWq8X58+eRn59v3iYxMRFqtRqBgYE1idOo2draYtSoUXBxccHatWt5mTaRFRFCICcnB1u3bkVkZCR8fX15E0Ci36lRQYmNjcWGDRuwadMmNGnSBLm5ucjNzTXPcHrlyhX84x//wOnTp3Ht2jXs3LkTU6dORd++fRESEgIAGDhwIAIDAzFlyhT88MMP2Lt3L5YsWYLY2FhelVJDcrkczzzzDNLT03HgwAGp4xDRAzKZTPjnP/8JJycn9OnTh+WE6B5qdJnx/f4SrV27Fk888QSysrIwefJkXLhwAaWlpfD29saoUaOwZMmSaqdlrl+/jjlz5uDQoUNwdHTEtGnT8Prrr8PG5sGGxDTWy4zvRQiB69ev46OPPsLkyZN5FIrICmzcuBE//vgjXn31VamjENWrmnx/P9I8KFJhQamuqqoKhw8fxtmzZzF16lS4urpKHYmI7mP//v1ITk7Gs88+ywHu1OjU2zwoZBlsbGzQu3dvyOVyfPnllzAYDFJHIqLfEULg0qVLOHjwIKZMmYKmTZtKHYnIorGgNBBKpRLz58/HoUOHcPz4cd5UkMjC6HQ6fPnll+jXrx/8/f057oToT7CgNCAymQzvvfce1q1bh4sXL0odh4j+q7KyEt988w0cHBzQr18/lhOiB8CC0sC4ublh3rx52LhxI37++Wep4xA1ekIIbNy4EYcOHcK4ceN4tSLRA2JBaYCCg4MRERGBzZs3N9r7FhFZikuXLuGbb77BwoULeZ8dohpgQWmAFAoFIiIioNFosGvXLs40SySR0tJSLFiwACtXrkS7du2kjkNkVVhQGihbW1v85S9/QVpaGo4cOcKSQlTPSkpK8Pbbb+OZZ5657208iOj+WFAaKJlMBjc3N8TExGDNmjVIS0uTOhJRo1FeXo7du3fDy8sLPXr0gEKhkDoSkdVhQWngevbsiRkzZuDtt9+GTqeTOg5Rg2cymZCamor09HQMHjwYTZo0kToSkVViQWkEBgwYgHHjxmHp0qWcH4WojlVUVODVV1/F1KlT4eXlJXUcIqvFgtJIREREoGPHjli3bh3vfExUR8rLyzFmzBg8/fTT8PHxkToOkVVjQWkEZDIZ7OzsMHToUNy4cQMpKSkwGo1SxyJqUPR6PVauXIkZM2bgscce42RsRI+IBaUR8fT0xKBBg7B//35cv36dp3uIakl5eTl27dqFpk2bIiYmhuWEqBawoDQywcHB6NWrF1555RUeRSGqBUIIHD9+HNeuXcPYsWNhZ2cndSSiBoEFpRHq378/hgwZghdeeIFHUYgegRAC2dnZ2LFjB/7yl7+gWbNmUkciajBYUBohW1tbjB07Fq1atcInn3yCyspKqSMRWaVbt25h+fLlGD16NPz9/aWOQ9SgsKA0UjY2Npg0aRIKCwtx8OBBlhSiGtLr9Vi4cCFcXV3Rp08fjjshqmUsKI2YRqPB448/jpSUFGRkZPB0D9EDMhgM+PzzzxEaGoqXXnpJ6jhEDRILSiPn5+eHQYMG4bPPPuNMs0QPaNeuXTAYDJg2bRrkcv4zSlQX+DeL0K1bNwQFBWHu3Lm8sofoDwghcObMGaSlpWHs2LFwcHCQOhJRg8WCQlAoFJgyZQrat2+PV155BeXl5VJHIrI4Qgj8+uuv2L59O2JiYtCyZUuOOyGqQywoBACQy+WIi4tDkyZN8PXXX6OiokLqSEQWJS8vDx9++CF69OiBrl27spwQ1TEWFDKzsbHBrFmzcPXqVRw+fJiDZon+q6ysDG+88Qbatm2LoUOHSh2HqFFgQaFqmjZtiscffxzJycn46aefpI5DZBE++OADhISE4IknnpA6ClGjwYJCd2nVqhVGjhyJ+Ph43Lp1S+o4RJIxmUz46quvYGdnhzFjxvCKHaJ6xL9tdBe5XI7Q0FCEh4dj4cKFuHnzptSRiOqdyWTCqVOnkJ6ejlGjRqFJkyYcd0JUj1hQ6J5kMhkmTJiAdu3aYeXKlZwjhRoVIQQuX76M3bt3Y8iQIbxih0gCLCj0h5599ln4+flh69atvPyYGo3MzEy89dZbGDhwIDp37ix1HKJGiQWF/pCtrS3GjRuH0tJS7N69m1f2UIN3+/ZtLF68GNOmTYNWq5U6DlGjxYJCf8rR0RFTpkzB999/j3PnzrGkUINVWVmJV199FTNnzkSPHj2kjkPUqNWooMTHxyMkJARqtRpqtRparRa7d+82ry8vL0dsbCyaNWsGJycnjBkzBnl5edVeIzMzE0OHDoWDgwPc3NywcOFCVFVV1c6noTrj4uKCuXPnYtWqVUhPT5c6DlGtKy8vx6ZNm9C+fXvenZjIAtSooLRs2RKvv/46Tp8+jVOnTmHAgAEYMWIE0tLSAPw2XmHXrl344osvkJycjOzsbIwePdr8fKPRiKFDh8JgMODo0aNYv3491q1bh5dffrl2PxXVOplMhlatWmHy5Ml47bXXcObMGakjEdWaqqoq7Nu3DzqdDjExMbCzs2NBIZKYTDzi8XoXFxesWLECY8eOhaurKzZt2oSxY8cCANLT09GhQwekpKSgR48e2L17N2JiYpCdnQ13d3cAwOrVq7F48WIUFBRAqVQ+0Hvq9XpoNBrodDqo1epHiU81JITAnj17cPjwYcyaNQutW7eWOhLRI9u3bx9OnTqFJ554Al5eXlLHIWqwavL9/dBjUIxGIzZv3ozS0lJotVqcPn0alZWViIqKMm8TEBAAHx8fpKSkAABSUlIQHBxsLicAEB0dDb1ebz4Kcy8VFRXQ6/XVFpKGTCbDwIEDMXDgQGzbtg0FBQUck0JWSwiBb775BmvWrMHMmTNZTogsSI0Lyvnz5+Hk5ASVSoWnnnoK27dvR2BgIHJzc6FUKuHs7Fxte3d3d+Tm5gIAcnNzq5WTO+vvrLuf5cuXQ6PRmBdvb++axqZapFAo0LdvX/j6+uLLL79EaWkpSwpZHZPJhLNnz2LTpk348MMP4ebmJnUkIvofNS4o7du3R2pqKo4fP445c+Zg2rRpuHjxYl1kM4uLi4NOpzMvWVlZdfp+9OcUCoX58uNt27bBZDJJHYnogQkhcPXqVWzfvh0vvfQSPDw8OOaEyMLUuKAolUq0bdsWoaGhWL58OTp16oR3330XHh4eMBgMKCoqqrZ9Xl4ePDw8AAAeHh53XdVz5+c729yLSqUyXzl0ZyHL8Nxzz+HHH3/EZ599JnUUogd248YNbNiwAYMGDUJAQIDUcYjoHh55HhSTyYSKigqEhobC1tYWSUlJ5nUZGRnIzMw0T3ak1Wpx/vx55Ofnm7dJTEyEWq1GYGDgo0Yhibz44ovIysrCRx99JHUUoj9VWVmJ119/HVqtFj179pQ6DhHdR40KSlxcHA4fPoxr167h/PnziIuLw6FDhzBp0iRoNBrMnDkTCxYswMGDB3H69GlMnz4dWq3WPOHRwIEDERgYiClTpuCHH37A3r17sWTJEsTGxkKlUtXJB6S616RJE8TGxqK8vBxfffUVx6OQxRJCYO7cuRgyZAgiIyN5WofIgtWooOTn52Pq1Klo3749IiMjcfLkSezduxePPfYYAOCdd95BTEwMxowZg759+8LDwwNfffWV+fkKhQIJCQlQKBTQarWYPHkypk6dir///e+1+6moXslkMri4uGDcuHG4dOkSvvvuOxiNRqljEVVz+/ZtzJgxA/7+/ujfvz8UCoXUkYjoDzzyPChS4Dwoluvq1avmc/vdunXjf6GSRdDpdNi6dSucnZ0xYsSIB55ziYhqV73Mg0J0L35+fvjLX/6CnTt34vvvv5c6DhFu376NnTt3wsHBAdHR0SwnRFaCBYVqXUBAACZMmID4+Hjs2rVL6jjUiJlMJmzatAmVlZUYOnQoj7gSWREWFKoTHTp0QFxcHE6dOsU7IJMkTCYT1q1bh5KSEowbN+6uSSSJyLLZSB2AGiaZTIagoCDzVOIqlQr+/v6Qy9mJqe6VlZVhzZo1SEtLw6pVq2Brayt1JCKqIX5bUJ2RyWQIDg5Gv379sHPnTly+fJlHUqjO3b59G99++y3Kysrw5ptvspwQWSkWFKpzWq0WvXv3xpYtW3DixAmp41ADZjAYsH//fhQWFmLGjBlo2rSp1JGI6CGxoFC90Gq1GDZsGN56661qsw0T1RYhBL7++mtcv34dI0aMuOvGpERkXVhQqN506tQJL774Ig4ePIj09HSe7qFaU1VVhQ0bNuCnn37CjBkzeGdiogaAg2Sp3shkMoSEhMBoNGL79u0YNWoU2rVrx4Gz9EhKS0vxwQcfoKioCMuWLeM8J0QNBL8ZqF7JZDJ07doVERER+Prrr3H+/HmpI5EVKykpwddffw2TyYTnnnuO5YSoAeERFJKEVquFnZ0dEhISkJ2djcGDB0sdiayMwWDAF198AQCYOXMmmjdvLnEiIqpNLCgkmc6dO8Pe3h5vvvkmhBAYMmSI1JHIirzzzjvw8PDAiBEjOAkbUQPEUzwkGZlMhvbt22PRokVISkrC4cOHYTKZpI5FFq68vBxLliyBj48PJkyYwHJC1ECxoJCk7pSUp59+GomJiUhOTmZJofu6desWlixZgjZt2mDs2LEcc0LUgLGgkORkMhl8fX0xZ84cJCcn46uvvpI6ElkYIQRycnLwySefICAgAKNHj+YMsUQNHAsKWQwvLy/Mnz8fOTk5WLFiBUpLS6WORBYiMzMTK1asQKdOnTB16lRoNBqpIxFRHWNBIYui0WgwY8YMNGvWDP/6179QVFQk6YRuQoi7Fqo/QgicP38eS5cuxaxZs/DYY4/xtA5RI8GCQhZFJpPBwcEBU6dOhb+/P+Lj41FQUCBJMRBC4NNPP0VgYCCeeOIJxMfH4/Tp0ygsLERxcTHKyspQVVVV77kai6qqKmzZsgVvvvkmXnvtNQQEBEAmk0kdi4jqCS8zJosjk8lgY2ODiRMn4ssvv8SqVaswadIk+Pv712uO4uJiXLx4Eenp6UhPT8dnn30GAGjatCk6deqEkJAQdO7cGe3atYNGozEvTZo04RfpI6qoqDBf2bVo0SJ4eXlJHYmI6plMWOExa71eD41GA51OB7VaLXUcqmOHDx/GgQMH0KdPH0RGRtbb+16+fBnjxo3DmTNn/nA7Ozs7tGnTBq1btzb/b4sWLcyLl5cXp/OvAYPBgLVr16KsrAyjRo2Cr6+v1JGIqJbU5PubR1DI4vXp0wdNmzbF+vXrkZeXh9GjR8POzq7O3/fWrVt/Wk6A3+blSEtLQ1paGgBALpfDzc2t2hIQEID27dub/1elUtV1fKuUl5eH5cuXo1u3bvjLX/4CFxcXqSMRkUR4BIWsgslkwq+//oq1a9eiefPmmDFjRp2WFCEEjhw5gr59+9bK6zk4OMDBwQGOjo5wcHBAcHAwQkNDERoaiq5du0Kj0VQ7LdTYThHd2d+ffPIJZs2ahe7du3MwLFEDVJPvbxYUshpCCBiNRsTHxyM/Px8LFixA06ZN6+S9qqqqsHr1asybN69OXl8mk0Eul5uXdu3aITw8HGFhYQgLC4OXlxeUSiVUKhWUSiVsbBrmwU4hBEwmE06dOoV33nkHs2bNwoABAxpdQSNqLFhQqMHbuXMnDh8+jIkTJyIoKKjWT5mUlZVh3Lhx2LVrV62+7oPy9PREx44dERISgo4dO6Jv375o3bq1JFnqkk6nQ1JSElJSUjB9+nQEBgZKHYmI6hALCjV4d/6re/PmzQgNDcWIESPg5ORUa69/53fMUrz22muIi4uTOkatunz5Mnbu3AmDwYAnn3ySdyMmagRq8v3NSwvIKsnlcnTr1g3z5s3DzZs38d5776G8vLzWXl+n09Xaa9HdkpKSsHr1agQEBGDRokUsJ0R0FxYUslpyuRx+fn6YMWMGgoKCMHz4cFy5cqVWbjb4IFfv1Jfg4OB6vby6LhkMBmzcuBFvv/02YmJiMHDgQF6CTUT31DBH3lGj4uTkhGHDhqFr1674v//7PwwbNgwjR46Eo6PjQw+2nDVr1l2P3RmwqlAoAABGoxEVFRUwGAyPlP/PeHl51fskdbXNZDKhoKAAa9asQUVFBbZu3QoHBwcOhiWi+2JBoQZBLpfD29sb77zzDlasWIHMzEyMHj0a7dq1e6j/QjcajeY/K5VKeHl5ISQkBL6+vtBoNBBCoKioCD///DMuXLiA3NxcVFZW1uZHMnN0dKyzq5XqQ2FhIU6cOIF9+/ahV69eGDNmjNSRiMgKsKBQg+Ll5YW4uDjs3bsXmzdvRseOHTF27Ngavcb/3qBQpVKha9eu6NmzJ5ycnKr9F7+rqyuaN2+Odu3a4bvvvsP58+frrKRYIyEE8vPzsWrVKpSWlmLGjBm8SoeIHliN/tMyPj4eISEhUKvVUKvV0Gq12L17t3l9REQEZDJZteWpp56q9hqZmZkYOnQoHBwc4ObmhoULF/KGa1Sr3NzcMGXKFEycOBFZWVn461//ikuXLj3w8y9cuGD+nfTz80OfPn3ue38dmUyGpk2bon///vD09Ky1z3BH06ZN8fjjj9f669aHxMREPPPMM2jdujWee+45dOzYkeNNiOiB1egISsuWLfH666/D398fQgisX78eI0aMwNmzZxEUFATgt3P3f//7383PcXBwMP/ZaDRi6NCh8PDwwNGjR5GTk4OpU6fC1tYWr732Wi19JKLf+Pv7Y/bs2UhJScGzzz6Lp59+Gv3794eNjc0fjn1Yvnw5SkpK4OHhgZEjRz7QjLVNmjTB+PHjsWbNmnteASSTydC8eXN07doVvr6+cHR0RGlpKa5du4YzZ87g5s2b97xjs1qtxoABA2r2wSVkMpmg1+uxcuVKFBUV4d1330XTpk05KywR1ViNCsqwYcOq/fzPf/4T8fHxOHbsmLmgODg4wMPD457P37dvHy5evIj9+/fD3d0dnTt3xj/+8Q8sXrwYr7zyCv8Ro1olk8ng4OCAyMhIaDQavPbaazh27BimTJmCFi1a3Pf3rbi4GHK5HN27d6/RdPoODg7QarXYs2dPtcdVKhW6d++OPn36wNbW1vy4Wq2Gp6cnunXrhuTkZJw8efKuAbc2NjZWcQmuEALFxcU4ePAgPv74Yzz++OMYP348/04T0UN76OOtRqMRmzdvRmlpKbRarfnxjRs3onnz5ujYsSPi4uJw+/Zt87qUlBQEBwfD3d3d/Fh0dDT0er35Rmv3UlFRAb1eX20hehB3TjWGhYVh+/btCAwMxLvvvott27bhl19+uedzHnvsMQwZMgRdunSp8fuFhYXddXSmZ8+eGDBgAJRK5V2nQGUyGZRKJaKiotCzZ8+7Xq9Dhw4Wf6WL0WjEqVOn8PHHH2Pv3r2Ii4vD1KlTWU6I6JHUeJDs+fPnodVqUV5eDicnJ/M/+gAwceJE+Pr6wsvLC+fOncPixYuRkZGBr776CgCQm5tbrZwAMP+cm5t73/dcvnw5li1bVtOoRHcZM2YMwsLCkJCQgLfeegu9evXCsGHDqk2V/9JLLyE7Oxsff/zxI79fWFgYevbs+aclQyaToWfPnigpKcGpU6cA/HZl0syZMx85Q126evUqtmzZAoPBgM6dO2P69Om8AzER1YoaF5T27dsjNTUVOp0O27Ztw7Rp05CcnIzAwEDMnj3bvF1wcDA8PT0RGRmJK1euoE2bNg8dMi4uDgsWLDD/rNfr4e3t/dCvR42XTCaDr68vpk+fjgsXLmD//v2YNWsWnn76aXTr1s28XbNmzR75vdzc3NCtW7dqp3X+iFKpRGhoKK5fv46CggLIZDKEh4c/co66UFZWhrVr1+LEiRMYPnw4wsPD4eXlZfFHe4jIetS4oCiVSrRt2xYAEBoaipMnT+Ldd9/FmjVr7tr2zj+uly9fRps2beDh4YETJ05U2yYvLw8A7jtuBfjtHH5t3wyOGjcHBweEhYUhODgYGRkZePvtt9GiRQs8+eST8PHxAQAoFIpq86E8CBsbG4SGhiI9PR1dunSBq6trjZ7v7u6OwMBAJCcnA4BFjT8xmUwwGAw4fvw43n77bbRq1QqLFi1Cu3btGuzdlolIOo98zZ/JZEJFRcU916WmpgKA+fJLrVaL8+fPIz8/37xNYmIi1Go150egeieTyWBvb49OnTphzZo1CAoKwpw5c7Bq1SpcuXIF3bt3r/FrDh48GCdOnMCvv/6KJ554osaX1crlcvNMtZZyRMJkMiE/Px+HDx/GSy+9hC1btuD111/HypUr0aFDB5YTIqoTNfqXJS4uDoMHD4aPjw+Ki4uxadMmHDp0CHv37sWVK1ewadMmDBkyBM2aNcO5c+fw7LPPom/fvggJCQEADBw4EIGBgZgyZQrefPNN5ObmYsmSJYiNjeUREpLMnaIyefJkdOrUCYmJiVi/fj00Gg2Kiorg7Oz8QK/Tpk0bBAUFQSaTQa1WP/JYjMWLF0s+b0h2djaOHz+Oo0ePorS0FGPGjEHv3r3595WI6lyNCkp+fj6mTp2KnJwcaDQahISEYO/evXjssceQlZWF/fv3Y+XKlSgtLYW3tzfGjBmDJUuWmJ+vUCiQkJCAOXPmQKvVwtHREdOmTas2bwqRlIKDgxEcHIzMzEwcO3YMt2/fRklJCZycnP7weV5eXhg4cGC1eX8elkajgb29PcLDwyU7gpKdnY0dO3bg2rVr8PT0REREBLp27Vonk9EREd2LTNxrdigLp9frodFooNPpoFarpY5DDZTRaMSNGzeQkpKCc+fOwWQyQSaTmSdUu1MeWrRogYEDB6JFixbm0zMA8NNPP+Gbb76p0WXxzs7OCA8Ph52dHTp06FAvRyr+95+An3/+Ge+//z6uXr2KmJgY9OzZEz4+PmjSpEmd5yCihq8m3988eUx0HwqFAu7u7oiJiTGPLUlJSYHBYEB5eTmaNWuGbt26oVu3brCzs7vraEfbtm3h4eFRo4Li6emJ7t2718upHSEEysvLUV5ejtTUVHz++efIysrCoEGD8Pzzz6NZs2Z/OusuEVFd4REUohrKzs7GBx98gJMnTyIkJARhYWEICAhA06ZN0aJFi7sGjX766afIysr609f19fXFE088UUepfyOEwO3bt5Gbm4vc3FwkJibixIkT8PHxwYQJE9CrVy8OeiWiOlOT728WFKKHVFlZiRMnTuDYsWPIz89Hfn4+OnTogA4dOsDf3x++vr6wt7dHXl4e9u3bh2vXrsFkMt31OnK5HH5+fhg4cCDc3NzqJOvNmzdx/vx5XL16FQUFBSgpKTGPFevXr99DzZpLRFRTLChE9chkMuHGjRs4e/asuajk5uaipKQE9vb2iIiIgLe3N0pKSnDlyhXk5eWhvLwcdnZ28PDwQPv27dG+fXs4OzvXyukUIQQqKyuRmpqK9PR0pKam4vbt2ygtLYW/vz+6du2KNm3awM/PDyqViqdwiKjesKAQSeTOTfP0ej1u3LiBLVu2oLS0FD///DNu3rwJPz8/2Nvbo1+/fggMDISvry+cnZ3Ng2sftCz8719bg8GAixcv4tKlS7h48SLS0tJw6dIl+Pj4mI+OeHt7w97eHs7OzhzwSkSSYUEhsgBCCBiNRgghYDKZoNPp8MMPP2DTpk0wGAzIycnBzZs30bRpU1RUVCAoKAhubm5wcnKCk5MT8vLy4OHhAaVSicrKSlRWViItLQ1KpRLl5eXIz89HXl4edDodvL29ERYWhqCgIAQFBaFdu3ZQKpWQy+WQy+XmGxMSEUmJBYXIShgMBuTl5eHYsWOwsbGB0WhEaWkpSkpKkJGRARcXF2g0Gtja2sLGxga5ubnw8/ND69at4erqCldXVzg7O5tLCBGRJeNlxkRWQqlUwtvbmze/JCL6HWnn0SYiIiK6BxYUIiIisjgsKERERGRxWFCIiIjI4rCgEBERkcVhQSEiIiKLw4JCREREFocFhYiIiCwOCwoRERFZHBYUIiIisjgsKERERGRxWFCIiIjI4rCgEBERkcVhQSEiIiKLw4JCREREFocFhYiIiCwOCwoRERFZHBYUIiIisjgsKERERGRxWFCIiIjI4rCgEBERkcVhQSEiIiKLw4JCREREFocFhYiIiCwOCwoRERFZHBupAzwMIQQAQK/XS5yEiIiIHtSd7+073+N/xCoLSnFxMQDA29tb4iRERERUU8XFxdBoNH+4jUw8SI2xMCaTCRkZGQgMDERWVhbUarXUkayWXq+Ht7c392Mt4L6sPdyXtYP7sfZwX9YOIQSKi4vh5eUFufyPR5lY5REUuVyOFi1aAADUajV/WWoB92Pt4b6sPdyXtYP7sfZwXz66PztycgcHyRIREZHFYUEhIiIii2O1BUWlUmHp0qVQqVRSR7Fq3I+1h/uy9nBf1g7ux9rDfVn/rHKQLBERETVsVnsEhYiIiBouFhQiIiKyOCwoREREZHFYUIiIiMjiWGVB+fDDD9GqVSvY2dkhPDwcJ06ckDqSxTl8+DCGDRsGLy8vyGQy7Nixo9p6IQRefvlleHp6wt7eHlFRUbh06VK1bQoLCzFp0iSo1Wo4Oztj5syZKCkpqcdPIb3ly5cjLCwMTZo0gZubG0aOHImMjIxq25SXlyM2NhbNmjWDk5MTxowZg7y8vGrbZGZmYujQoXBwcICbmxsWLlyIqqqq+vwokoqPj0dISIh5kiutVovdu3eb13MfPrzXX38dMpkM8+fPNz/G/flgXnnlFchksmpLQECAeT33o8SEldm8ebNQKpXi008/FWlpaWLWrFnC2dlZ5OXlSR3Nonz77bfixRdfFF999ZUAILZv315t/euvvy40Go3YsWOH+OGHH8Tw4cOFn5+fKCsrM28zaNAg0alTJ3Hs2DHx3XffibZt24oJEybU8yeRVnR0tFi7dq24cOGCSE1NFUOGDBE+Pj6ipKTEvM1TTz0lvL29RVJSkjh16pTo0aOH6Nmzp3l9VVWV6Nixo4iKihJnz54V3377rWjevLmIi4uT4iNJYufOneKbb74RP/30k8jIyBAvvPCCsLW1FRcuXBBCcB8+rBMnTohWrVqJkJAQ8cwzz5gf5/58MEuXLhVBQUEiJyfHvBQUFJjXcz9Ky+oKSvfu3UVsbKz5Z6PRKLy8vMTy5cslTGXZfl9QTCaT8PDwECtWrDA/VlRUJFQqlfjPf/4jhBDi4sWLAoA4efKkeZvdu3cLmUwmfv3113rLbmny8/MFAJGcnCyE+G2/2draii+++MK8zY8//igAiJSUFCHEb2VRLpeL3Nxc8zbx8fFCrVaLioqK+v0AFqRp06bik08+4T58SMXFxcLf318kJiaKfv36mQsK9+eDW7p0qejUqdM913E/Ss+qTvEYDAacPn0aUVFR5sfkcjmioqKQkpIiYTLrcvXqVeTm5lbbjxqNBuHh4eb9mJKSAmdnZ3Tr1s28TVRUFORyOY4fP17vmS2FTqcDALi4uAAATp8+jcrKymr7MiAgAD4+PtX2ZXBwMNzd3c3bREdHQ6/XIy0trR7TWwaj0YjNmzejtLQUWq2W+/AhxcbGYujQodX2G8DfyZq6dOkSvLy80Lp1a0yaNAmZmZkAuB8tgVXdLPDGjRswGo3VfhkAwN3dHenp6RKlsj65ubkAcM/9eGddbm4u3Nzcqq23sbGBi4uLeZvGxmQyYf78+ejVqxc6duwI4Lf9pFQq4ezsXG3b3+/Le+3rO+sai/Pnz0Or1aK8vBxOTk7Yvn07AgMDkZqayn1YQ5s3b8aZM2dw8uTJu9bxd/LBhYeHY926dWjfvj1ycnKwbNky9OnTBxcuXOB+tABWVVCIpBQbG4sLFy7gyJEjUkexSu3bt0dqaip0Oh22bduGadOmITk5WepYVicrKwvPPPMMEhMTYWdnJ3UcqzZ48GDzn0NCQhAeHg5fX19s3boV9vb2EiYjwMqu4mnevDkUCsVdo6jz8vLg4eEhUSrrc2df/dF+9PDwQH5+frX1VVVVKCwsbJT7eu7cuUhISMDBgwfRsmVL8+MeHh4wGAwoKiqqtv3v9+W99vWddY2FUqlE27ZtERoaiuXLl6NTp0549913uQ9r6PTp08jPz0fXrl1hY2MDGxsbJCcn47333oONjQ3c3d25Px+Ss7Mz2rVrh8uXL/P30gJYVUFRKpUIDQ1FUlKS+TGTyYSkpCRotVoJk1kXPz8/eHh4VNuPer0ex48fN+9HrVaLoqIinD592rzNgQMHYDKZEB4eXu+ZpSKEwNy5c7F9+3YcOHAAfn5+1daHhobC1ta22r7MyMhAZmZmtX15/vz5aoUvMTERarUagYGB9fNBLJDJZEJFRQX3YQ1FRkbi/PnzSE1NNS/dunXDpEmTzH/m/nw4JSUluHLlCjw9Pfl7aQmkHqVbU5s3bxYqlUqsW7dOXLx4UcyePVs4OztXG0VNv43wP3v2rDh79qwAIN5++21x9uxZcf36dSHEb5cZOzs7i6+//lqcO3dOjBgx4p6XGXfp0kUcP35cHDlyRPj7+ze6y4znzJkjNBqNOHToULVLEW/fvm3e5qmnnhI+Pj7iwIED4tSpU0Kr1QqtVmtef+dSxIEDB4rU1FSxZ88e4erq2qguRXz++edFcnKyuHr1qjh37px4/vnnhUwmE/v27RNCcB8+qv+9ikcI7s8H9dxzz4lDhw6Jq1eviu+//15ERUWJ5s2bi/z8fCEE96PUrK6gCCHE+++/L3x8fIRSqRTdu3cXx44dkzqSxTl48KAAcNcybdo0IcRvlxq/9NJLwt3dXahUKhEZGSkyMjKqvcbNmzfFhAkThJOTk1Cr1WL69OmiuLhYgk8jnXvtQwBi7dq15m3KysrE3/72N9G0aVPh4OAgRo0aJXJycqq9zrVr18TgwYOFvb29aN68uXjuuedEZWVlPX8a6cyYMUP4+voKpVIpXF1dRWRkpLmcCMF9+Kh+X1C4Px/MuHHjhKenp1AqlaJFixZi3Lhx4vLly+b13I/SkgkhhDTHboiIiIjuzarGoBAREVHjwIJCREREFocFhYiIiCwOCwoRERFZHBYUIiIisjgsKERERGRxWFCIiIjI4rCgEBERkcVhQSEiIiKLw4JCREREFocFhYiIiCwOCwoRERFZnP8HFzehW/D8Q4AAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "env = gym.make('MountainCar-v0', render_mode=\"rgb_array\")\n",
    "# reset environment and get initial observation/state\n",
    "# Observation/state is a tuple of (position, velocity)\n",
    "obs, info = env.reset()\n",
    "print(\"initial observation:\", obs)\n",
    "\n",
    "# possible 3 actions\n",
    "# {0: \"accelerate to left\", \"1\": \"do nothing\", \"2\": \"accelerate to right\"}\n",
    "print(\"possible actions:\", env.action_space.n)\n",
    "\n",
    "\n",
    "# reinforcement learning is all\n",
    "# about learning to take good actions\n",
    "# from a given state/observation\n",
    "# right now taking a random action\n",
    "def policy(observation):\n",
    "    return env.action_space.sample()\n",
    "\n",
    "\n",
    "# take 5 random actions/steps\n",
    "for _ in range(5):\n",
    "\n",
    "    # to render environment for visual inspection\n",
    "    # when you train, you can skip rendering to speed up\n",
    "    plt.imshow(env.render())\n",
    "\n",
    "    # based on current policy, use the current observation\n",
    "    # to find the best action to take.\n",
    "    action = policy(obs)\n",
    "    print(\"\\ntaking action:\", action)\n",
    "\n",
    "    # pass the action to env which will return back\n",
    "    # with new state/\"observation\" and \"reward\"\n",
    "    # there is a \"done\" flag which is true when game ends\n",
    "    # \"info\" provides some diagnostic information\n",
    "    obs, reward, terminated, truncated, info = env.step(action)\n",
    "    print(\"got reward: {0}. New state/observation is: {1}\".format(reward, obs))\n",
    "\n",
    "# close the environment\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bmJ_FbX5ycEg"
   },
   "source": [
    "#### Second Environment Cart Pole\n",
    "\n",
    "This [environment](https://gymnasium.farama.org/environments/classic_control/cart_pole/) corresponds to the version of the cart-pole problem described by Barto, Sutton, and Anderson in “ Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problem ”. A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. The pendulum is placed upright on the cart and the goal is to balance the pole by applying forces in the left and right direction on the cart.\n",
    "\n",
    "**Action Space**\n",
    "The action is a ndarray with shape (1,) which can take values {0, 1} indicating the direction of the fixed force the cart is pushed with. `0` is the action to push the cart to the left and `1` is the action to push the cart to the right.\n",
    "\n",
    "**Observation Space**\n",
    "The observation is a ndarray with shape (4,) with the values corresponding to the following positions and velocities:\n",
    "\n",
    "| Num |      Observation      |         Min         |        Max        |\n",
    "|:---:|:---------------------:|:-------------------:|:-----------------:|\n",
    "| 0   | Cart Position         | -4.8                | 4.8               |\n",
    "| 1   | Cart Velocity         | -Inf                | Inf               |\n",
    "| 2   | Pole Angle            | ~ -0.418 rad (-24°) | ~ 0.418 rad (24°) |\n",
    "| 3   | Pole Angular Velocity | -Inf                | Inf               |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 691
    },
    "id": "GVNZ2YqHycEh",
    "outputId": "48554deb-83bd-429d-b1ce-d0f775cdae87"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "initial observation: (array([ 0.018456  , -0.0142253 ,  0.04051789,  0.00331859], dtype=float32), {})\n",
      "possible actions: 2\n",
      "\n",
      "taking action: 1\n",
      "got reward: 1.0. New state/observation is: [ 0.0181715   0.18029284  0.04058427 -0.27631015]\n",
      "\n",
      "taking action: 1\n",
      "got reward: 1.0. New state/observation is: [ 0.02177735  0.374813    0.03505806 -0.5559216 ]\n",
      "\n",
      "taking action: 0\n",
      "got reward: 1.0. New state/observation is: [ 0.02927361  0.17921682  0.02393963 -0.2524027 ]\n",
      "\n",
      "taking action: 1\n",
      "got reward: 1.0. New state/observation is: [ 0.03285795  0.3739889   0.01889157 -0.5374395 ]\n",
      "\n",
      "taking action: 0\n",
      "got reward: 1.0. New state/observation is: [ 0.04033773  0.1786065   0.00814279 -0.2388644 ]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAF7CAYAAAD4/3BBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/SrBM8AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAoIUlEQVR4nO3df3BUVZ7//1d3fjSE0B0DJJ1IgigMECE4Cxh6nHGZIUOA6MoYa0VZiLMUlGxiDcRhMLOOiLNlXNxaf8wq/DG74n5KBodZ0ZURnBgkrGP4YYYsvzQrfJgNLumEkU+6ASUh6fP9wy9d24pAh5A+3TwfVbcqfc/p7vc9lU5ede65tx3GGCMAAACLOGNdAAAAwJcRUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdWIaUF544QXdcMMNGjBggIqKirR79+5YlgMAACwRs4Dy6quvqqqqSitXrtQf/vAHTZw4USUlJWpvb49VSQAAwBKOWH1ZYFFRkaZMmaJ/+qd/kiSFQiHl5eXpoYce0iOPPBKLkgAAgCWSY/GmXV1damxsVHV1dXif0+lUcXGxGhoavtK/s7NTnZ2d4cehUEgnT57UkCFD5HA4+qVmAABwZYwxOnXqlHJzc+V0XvwkTkwCyp/+9Cf19PQoOzs7Yn92drY++uijr/SvqanRqlWr+qs8AABwFR07dkzDhw+/aJ+YBJRoVVdXq6qqKvw4EAgoPz9fx44dk9vtjmFlAADgcgWDQeXl5Wnw4MGX7BuTgDJ06FAlJSWpra0tYn9bW5u8Xu9X+rtcLrlcrq/sd7vdBBQAAOLM5SzPiMlVPKmpqZo0aZLq6urC+0KhkOrq6uTz+WJREgAAsEjMTvFUVVWpvLxckydP1q233qpnn31WZ86c0Q9/+MNYlQQAACwRs4By77336sSJE3rsscfk9/t1yy23aOvWrV9ZOAsAAK49MbsPypUIBoPyeDwKBAKsQQEAIE5E8/+b7+IBAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALBOnweUxx9/XA6HI2IbO3ZsuP3s2bOqqKjQkCFDlJ6errKyMrW1tfV1GQAAII5dlRmUm2++Wa2treHtvffeC7ctW7ZMb775pjZu3Kj6+nodP35cd99999UoAwAAxKnkq/Kiycnyer1f2R8IBPTP//zPWr9+vb73ve9Jkl566SWNGzdOO3fu1NSpU69GOQAAIM5clRmUjz/+WLm5ubrxxhs1b948tbS0SJIaGxt17tw5FRcXh/uOHTtW+fn5amho+NrX6+zsVDAYjNgAAEDi6vOAUlRUpHXr1mnr1q1as2aNjh49qu985zs6deqU/H6/UlNTlZGREfGc7Oxs+f3+r33NmpoaeTye8JaXl9fXZQMAAIv0+SmeWbNmhX8uLCxUUVGRRowYoV//+tcaOHBgr16zurpaVVVV4cfBYJCQAgBAArvqlxlnZGToG9/4hg4fPiyv16uuri51dHRE9Glra7vgmpXzXC6X3G53xAYAABLXVQ8op0+f1pEjR5STk6NJkyYpJSVFdXV14fbm5ma1tLTI5/Nd7VIAAECc6PNTPD/+8Y915513asSIETp+/LhWrlyppKQk3XffffJ4PFq4cKGqqqqUmZkpt9uthx56SD6fjyt4AABAWJ8HlE8++UT33XefPv30Uw0bNkzf/va3tXPnTg0bNkyS9Mwzz8jpdKqsrEydnZ0qKSnRiy++2NdlAACAOOYwxphYFxGtYDAoj8ejQCDAehQAAOJENP+/+S4eAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1og4oO3bs0J133qnc3Fw5HA69/vrrEe3GGD322GPKycnRwIEDVVxcrI8//jiiz8mTJzVv3jy53W5lZGRo4cKFOn369BUdCAAASBxRB5QzZ85o4sSJeuGFFy7Yvnr1aj3//PNau3atdu3apUGDBqmkpERnz54N95k3b54OHjyo2tpabd68WTt27NDixYt7fxQAACChOIwxptdPdji0adMmzZkzR9IXsye5ubl6+OGH9eMf/1iSFAgElJ2drXXr1mnu3Ln68MMPVVBQoD179mjy5MmSpK1bt2r27Nn65JNPlJube8n3DQaD8ng8CgQCcrvdvS0fAAD0o2j+f/fpGpSjR4/K7/eruLg4vM/j8aioqEgNDQ2SpIaGBmVkZITDiSQVFxfL6XRq165dF3zdzs5OBYPBiA0AACSuPg0ofr9fkpSdnR2xPzs7O9zm9/uVlZUV0Z6cnKzMzMxwny+rqamRx+MJb3l5eX1ZNgAAsExcXMVTXV2tQCAQ3o4dOxbrkgAAwFXUpwHF6/VKktra2iL2t7W1hdu8Xq/a29sj2ru7u3Xy5Mlwny9zuVxyu90RGwAASFx9GlBGjhwpr9erurq68L5gMKhdu3bJ5/NJknw+nzo6OtTY2Bjus23bNoVCIRUVFfVlOQAAIE4lR/uE06dP6/Dhw+HHR48eVVNTkzIzM5Wfn6+lS5fq7/7u7zR69GiNHDlSP/vZz5Sbmxu+0mfcuHGaOXOmFi1apLVr1+rcuXOqrKzU3LlzL+sKHgAAkPiiDigffPCBvvvd74YfV1VVSZLKy8u1bt06/eQnP9GZM2e0ePFidXR06Nvf/ra2bt2qAQMGhJ/zyiuvqLKyUtOnT5fT6VRZWZmef/75PjgcAACQCK7oPiixwn1QAACIPzG7DwoAAEBfIKAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALBO1AFlx44duvPOO5WbmyuHw6HXX389ov2BBx6Qw+GI2GbOnBnR5+TJk5o3b57cbrcyMjK0cOFCnT59+ooOBAAAJI6oA8qZM2c0ceJEvfDCC1/bZ+bMmWptbQ1vv/rVryLa582bp4MHD6q2tlabN2/Wjh07tHjx4uirBwAACSk52ifMmjVLs2bNumgfl8slr9d7wbYPP/xQW7du1Z49ezR58mRJ0i9+8QvNnj1b//AP/6Dc3NxoSwIAAAnmqqxB2b59u7KysjRmzBgtWbJEn376abitoaFBGRkZ4XAiScXFxXI6ndq1a9cFX6+zs1PBYDBiAwAAiavPA8rMmTP1r//6r6qrq9Pf//3fq76+XrNmzVJPT48kye/3KysrK+I5ycnJyszMlN/vv+Br1tTUyOPxhLe8vLy+LhsAAFgk6lM8lzJ37tzwzxMmTFBhYaFuuukmbd++XdOnT+/Va1ZXV6uqqir8OBgMElIAAEhgV/0y4xtvvFFDhw7V4cOHJUler1ft7e0Rfbq7u3Xy5MmvXbficrnkdrsjNgAAkLiuekD55JNP9OmnnyonJ0eS5PP51NHRocbGxnCfbdu2KRQKqaio6GqXAwAA4kDUp3hOnz4dng2RpKNHj6qpqUmZmZnKzMzUqlWrVFZWJq/XqyNHjugnP/mJRo0apZKSEknSuHHjNHPmTC1atEhr167VuXPnVFlZqblz53IFDwAAkCQ5jDEmmids375d3/3ud7+yv7y8XGvWrNGcOXO0d+9edXR0KDc3VzNmzNDPf/5zZWdnh/uePHlSlZWVevPNN+V0OlVWVqbnn39e6enpl1VDMBiUx+NRIBDgdA8AAHEimv/fUQcUGxBQAACIP9H8/+a7eAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOlF/WSAAXG1dZzr0xx3/56J9nMmpuql4sRwORz9VBaA/EVAAWMUYo55znQq07L9ov6TUgf1UEYBY4BQPAOuYUE+sSwAQYwQUANYhoAAgoACwjwnFugIAMUZAAWAdEyKgANc6AgoA6xjDKR7gWkdAAWAd1qAAIKAAsA6neAAQUABYhxkUAAQUAPbhKh7gmkdAAWAdZlAAEFAAWIc1KAAIKACsw2XGAAgoAKzDDAoAAgoA6zCDAoCAAsA+LJIFrnkEFADW4RQPAAIKAOtwmTEAAgoA6xhu1AZc86IKKDU1NZoyZYoGDx6srKwszZkzR83NzRF9zp49q4qKCg0ZMkTp6ekqKytTW1tbRJ+WlhaVlpYqLS1NWVlZWr58ubq7u6/8aAAkBE7xAIgqoNTX16uiokI7d+5UbW2tzp07pxkzZujMmTPhPsuWLdObb76pjRs3qr6+XsePH9fdd98dbu/p6VFpaam6urr0/vvv6+WXX9a6dev02GOP9d1RAYhrnOIB4DDGmN4++cSJE8rKylJ9fb1uv/12BQIBDRs2TOvXr9c999wjSfroo480btw4NTQ0aOrUqdqyZYvuuOMOHT9+XNnZ2ZKktWvXasWKFTpx4oRSU1Mv+b7BYFAej0eBQEBut7u35QOwkDEhHf/Db3X8gzcv2i8pdaC++cCzcjgc/VQZgCsVzf/vK1qDEggEJEmZmZmSpMbGRp07d07FxcXhPmPHjlV+fr4aGhokSQ0NDZowYUI4nEhSSUmJgsGgDh48eMH36ezsVDAYjNgAJC5O8QDodUAJhUJaunSpbrvtNo0fP16S5Pf7lZqaqoyMjIi+2dnZ8vv94T7/O5ycbz/fdiE1NTXyeDzhLS8vr7dlA4gDBBQAvQ4oFRUVOnDggDZs2NCX9VxQdXW1AoFAeDt27NhVf08AscMaFADJvXlSZWWlNm/erB07dmj48OHh/V6vV11dXero6IiYRWlra5PX6w332b17d8Trnb/K53yfL3O5XHK5XL0pFUAc6j576pJ9klyD+qESALES1QyKMUaVlZXatGmTtm3bppEjR0a0T5o0SSkpKaqrqwvva25uVktLi3w+nyTJ5/Np//79am9vD/epra2V2+1WQUHBlRwLgERgjD79eNcluw0be1s/FAMgVqKaQamoqND69ev1xhtvaPDgweE1Ix6PRwMHDpTH49HChQtVVVWlzMxMud1uPfTQQ/L5fJo6daokacaMGSooKND8+fO1evVq+f1+Pfroo6qoqGCWBMBlczi4zySQyKIKKGvWrJEkTZs2LWL/Sy+9pAceeECS9Mwzz8jpdKqsrEydnZ0qKSnRiy++GO6blJSkzZs3a8mSJfL5fBo0aJDKy8v1xBNPXNmRALimOJxJsS4BwFV0RfdBiRXugwIkLhPq0Qe/rJAucbv7PN+9yp7wPe6DAsSRfrsPCgDECjMoQGIjoACISw4nf76ARMYnHEBcIqAAiY1POIC4xCkeILERUADEJQIKkNgIKADik4OAAiQyAgqAuMQaFCCx8QkHEJcIKEBi4xMOIC45WYMCJDQCCoD4xBoUIKERUADEJU7xAImNTziAuORgBgVIaAQUAHGJ+6AAiY2AAiAucYoHSGx8wgHEJWZQgMRGQAEQn5hBARIan3AAcYlFskBiI6AAiEusQQESG59wAHGJGRQgsRFQAMQlZlCAxMYnHEBc4ioeILERUABYxZjL6+dwOuVwOK5uMQBihoACwCrG9MS6BAAWIKAAsEsoFOsKAFiAgALAKsYQUAAQUABYxoQ4xQOAgALAMiYUknSZK2UBJCwCCgCrsEgWgERAAWAbFskCEAEFgGVYgwJAIqAAsAyneABIUQaUmpoaTZkyRYMHD1ZWVpbmzJmj5ubmiD7Tpk2Tw+GI2B588MGIPi0tLSotLVVaWpqysrK0fPlydXd3X/nRAIh7hlM8ACQlR9O5vr5eFRUVmjJlirq7u/XTn/5UM2bM0KFDhzRo0KBwv0WLFumJJ54IP05LSwv/3NPTo9LSUnm9Xr3//vtqbW3VggULlJKSoieffLIPDglAPOMUDwApyoCydevWiMfr1q1TVlaWGhsbdfvtt4f3p6Wlyev1XvA1fve73+nQoUN65513lJ2drVtuuUU///nPtWLFCj3++ONKTU3txWEASBTMoACQrnANSiAQkCRlZmZG7H/llVc0dOhQjR8/XtXV1frss8/CbQ0NDZowYYKys7PD+0pKShQMBnXw4MELvk9nZ6eCwWDEBiAxsQYFgBTlDMr/FgqFtHTpUt12220aP358eP/999+vESNGKDc3V/v27dOKFSvU3Nys1157TZLk9/sjwomk8GO/33/B96qpqdGqVat6WyqAOMIMCgDpCgJKRUWFDhw4oPfeey9i/+LFi8M/T5gwQTk5OZo+fbqOHDmim266qVfvVV1draqqqvDjYDCovLy83hUOwGrMoACQenmKp7KyUps3b9a7776r4cOHX7RvUVGRJOnw4cOSJK/Xq7a2tog+5x9/3boVl8slt9sdsQFITMygAJCiDCjGGFVWVmrTpk3atm2bRo4cecnnNDU1SZJycnIkST6fT/v371d7e3u4T21trdxutwoKCqIpB0ACMqEevooHQHSneCoqKrR+/Xq98cYbGjx4cHjNiMfj0cCBA3XkyBGtX79es2fP1pAhQ7Rv3z4tW7ZMt99+uwoLCyVJM2bMUEFBgebPn6/Vq1fL7/fr0UcfVUVFhVwuV98fIYD4wgwKAEU5g7JmzRoFAgFNmzZNOTk54e3VV1+VJKWmpuqdd97RjBkzNHbsWD388MMqKyvTm2++GX6NpKQkbd68WUlJSfL5fPqrv/orLViwIOK+KQCuXaxBASBFOYNizMXnXfPy8lRfX3/J1xkxYoTeeuutaN4awDWCNSgAJL6LB4BlCCgAJAIKAMtwq3sAEgEFgG1YgwJABBQAluEUDwCJgALAMsYQUAAQUABYpv1QvS51p7Yho4vkTErpn4IAxAQBBYBVTHfXJfs4kwdIclz9YgDEDAEFQPxx8qcLSHR8ygHEHYfTyQQKkOAIKADijsPBny4g0fEpBxB3HI4kMYUCJDYCCoD4wxoUIOHxKQcQdxwEFCDh8SkHEHdYgwIkPj7lAOKOw5kU6xIAXGUEFABx54sZFBbJAomMgAIg7jicTjnIJ0BCI6AAiD8OTvEAiY6AAiDucBUPkPiSY10AgMQRCoUUCoWu6DUu/j3G/38fI3V398jhvJzeF5aczJ8/wGZ8QgH0mX/7t3/T/ffff0WvsfHxe3T90MEX7fPgkiXasvuIQqHeBZRx48Zp3759vXougP5BQAHQZ0KhkLq7u6/oNYy5dOjoOtet7nPdCl1G3wvp6enp1fMA9B8CCgArne1J04lzeTobGqQkdcuTfEJDUlslST29nDkBED8IKACsczaUpr2ninW6J0PdxiWHejTQeUZ5Az7UjWn71NMTkrms1SoA4hUBBYBVQiZZ73f8QJ2hQeF9Rsn6LOTRx59NVoqzS92h2hhWCKA/cK0eAKu813GPOkNpF2wLKVkHTn9HJzuH9XNVAPobAQWAVb44cXOx28Q61B0yl3c9MoC4RUABEHd6QiHyCZDgCCgA4k5PD/EESHQEFABW+ZbndSU7ui7Y5lBIYwc1aHBSWz9XBaC/RRVQ1qxZo8LCQrndbrndbvl8Pm3ZsiXcfvbsWVVUVGjIkCFKT09XWVmZ2toi/5C0tLSotLRUaWlpysrK0vLly6/4xk4AEkeK86y+nbFRg5L+n5LUJcnIoR65nGd008C9umHAfoW40RqQ8KK6zHj48OF66qmnNHr0aBlj9PLLL+uuu+7S3r17dfPNN2vZsmX67W9/q40bN8rj8aiyslJ33323fv/730v64u6NpaWl8nq9ev/999Xa2qoFCxYoJSVFTz755FU5QADx5Z0P/q8yBh/X5z3Nau26UZ/3uJXkOKfMlFYFU1t0UNKnwc9iXSaAq8xhLue+0heRmZmpp59+Wvfcc4+GDRum9evX65577pEkffTRRxo3bpwaGho0depUbdmyRXfccYeOHz+u7OxsSdLatWu1YsUKnThxQqmpqZf1nsFgUB6PRw888MBlPwfA1XfkyBHV1dXFuoxLysjI0F/+5V/GugzgmtPV1aV169YpEAjI7XZftG+vb9TW09OjjRs36syZM/L5fGpsbNS5c+dUXFwc7jN27Fjl5+eHA0pDQ4MmTJgQDieSVFJSoiVLlujgwYP65je/ecH36uzsVGdnZ/hxMBiUJM2fP1/p6em9PQQAfeydd96Ji4By3XXXaeHChbEuA7jmnD59WuvWrbusvlEHlP3798vn8+ns2bNKT0/Xpk2bVFBQoKamJqWmpiojIyOif3Z2tvx+vyTJ7/dHhJPz7efbvk5NTY1WrVr1lf2TJ0++ZAID0H+OHj0a6xIuy8CBA3XrrbfGugzgmnN+guFyRH0Vz5gxY9TU1KRdu3ZpyZIlKi8v16FDh6J9mahUV1crEAiEt2PHjl3V9wMAALEV9QxKamqqRo0aJUmaNGmS9uzZo+eee0733nuvurq61NHRETGL0tbWJq/XK0nyer3avXt3xOudv8rnfJ8Lcblccrlc0ZYKAADi1BXfByUUCqmzs1OTJk1SSkpKxPnn5uZmtbS0yOfzSZJ8Pp/279+v9vb2cJ/a2lq53W4VFBRcaSkAACBBRDWDUl1drVmzZik/P1+nTp3S+vXrtX37dr399tvyeDxauHChqqqqlJmZKbfbrYceekg+n09Tp06VJM2YMUMFBQWaP3++Vq9eLb/fr0cffVQVFRXMkAAAgLCoAkp7e7sWLFig1tZWeTweFRYW6u2339b3v/99SdIzzzwjp9OpsrIydXZ2qqSkRC+++GL4+UlJSdq8ebOWLFkin8+nQYMGqby8XE888UTfHhUAAIhrV3wflFg4fx+Uy7mOGkD/efXVVzV37txYl3FJBQUFOnjwYKzLAK450fz/5rt4AACAdQgoAADAOgQUAABgHQIKAACwTq+/iwcAvuz666/XnDlzYl3GJeXl5cW6BACXwFU8AACgX3AVDwAAiGsEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHWiCihr1qxRYWGh3G633G63fD6ftmzZEm6fNm2aHA5HxPbggw9GvEZLS4tKS0uVlpamrKwsLV++XN3d3X1zNAAAICEkR9N5+PDheuqppzR69GgZY/Tyyy/rrrvu0t69e3XzzTdLkhYtWqQnnngi/Jy0tLTwzz09PSotLZXX69X777+v1tZWLViwQCkpKXryySf76JAAAEC8cxhjzJW8QGZmpp5++mktXLhQ06ZN0y233KJnn332gn23bNmiO+64Q8ePH1d2drYkae3atVqxYoVOnDih1NTUy3rPYDAoj8ejQCAgt9t9JeUDAIB+Es3/716vQenp6dGGDRt05swZ+Xy+8P5XXnlFQ4cO1fjx41VdXa3PPvss3NbQ0KAJEyaEw4kklZSUKBgM6uDBg1/7Xp2dnQoGgxEbAABIXFGd4pGk/fv3y+fz6ezZs0pPT9emTZtUUFAgSbr//vs1YsQI5ebmat++fVqxYoWam5v12muvSZL8fn9EOJEUfuz3+7/2PWtqarRq1apoSwUAAHEq6oAyZswYNTU1KRAI6De/+Y3Ky8tVX1+vgoICLV68ONxvwoQJysnJ0fTp03XkyBHddNNNvS6yurpaVVVV4cfBYFB5eXm9fj0AAGC3qE/xpKamatSoUZo0aZJqamo0ceJEPffccxfsW1RUJEk6fPiwJMnr9aqtrS2iz/nHXq/3a9/T5XKFrxw6vwEAgMR1xfdBCYVC6uzsvGBbU1OTJCknJ0eS5PP5tH//frW3t4f71NbWyu12h08TAQAARHWKp7q6WrNmzVJ+fr5OnTql9evXa/v27Xr77bd15MgRrV+/XrNnz9aQIUO0b98+LVu2TLfffrsKCwslSTNmzFBBQYHmz5+v1atXy+/369FHH1VFRYVcLtdVOUAAABB/ogoo7e3tWrBggVpbW+XxeFRYWKi3335b3//+93Xs2DG98847evbZZ3XmzBnl5eWprKxMjz76aPj5SUlJ2rx5s5YsWSKfz6dBgwapvLw84r4pAAAAV3wflFjgPigAAMSffrkPCgAAwNVCQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArJMc6wJ6wxgjSQoGgzGuBAAAXK7z/7fP/x+/mLgMKKdOnZIk5eXlxbgSAAAQrVOnTsnj8Vy0j8NcToyxTCgUUnNzswoKCnTs2DG53e5YlxS3gsGg8vLyGMc+wFj2HcaybzCOfYex7BvGGJ06dUq5ublyOi++yiQuZ1CcTqeuv/56SZLb7eaXpQ8wjn2Hsew7jGXfYBz7DmN55S41c3Iei2QBAIB1CCgAAMA6cRtQXC6XVq5cKZfLFetS4hrj2HcYy77DWPYNxrHvMJb9Ly4XyQIAgMQWtzMoAAAgcRFQAACAdQgoAADAOgQUAABgnbgMKC+88IJuuOEGDRgwQEVFRdq9e3esS7LOjh07dOeddyo3N1cOh0Ovv/56RLsxRo899phycnI0cOBAFRcX6+OPP47oc/LkSc2bN09ut1sZGRlauHChTp8+3Y9HEXs1NTWaMmWKBg8erKysLM2ZM0fNzc0Rfc6ePauKigoNGTJE6enpKisrU1tbW0SflpYWlZaWKi0tTVlZWVq+fLm6u7v781Bias2aNSosLAzf5Mrn82nLli3hdsaw95566ik5HA4tXbo0vI/xvDyPP/64HA5HxDZ27NhwO+MYYybObNiwwaSmppp/+Zd/MQcPHjSLFi0yGRkZpq2tLdalWeWtt94yf/u3f2tee+01I8ls2rQpov2pp54yHo/HvP766+Y///M/zV/8xV+YkSNHms8//zzcZ+bMmWbixIlm586d5j/+4z/MqFGjzH333dfPRxJbJSUl5qWXXjIHDhwwTU1NZvbs2SY/P9+cPn063OfBBx80eXl5pq6uznzwwQdm6tSp5lvf+la4vbu724wfP94UFxebvXv3mrfeessMHTrUVFdXx+KQYuLf//3fzW9/+1vzX//1X6a5udn89Kc/NSkpKebAgQPGGMawt3bv3m1uuOEGU1hYaH70ox+F9zOel2flypXm5ptvNq2treHtxIkT4XbGMbbiLqDceuutpqKiIvy4p6fH5ObmmpqamhhWZbcvB5RQKGS8Xq95+umnw/s6OjqMy+Uyv/rVr4wxxhw6dMhIMnv27An32bJli3E4HOZ//ud/+q1227S3txtJpr6+3hjzxbilpKSYjRs3hvt8+OGHRpJpaGgwxnwRFp1Op/H7/eE+a9asMW6323R2dvbvAVjkuuuuM7/85S8Zw146deqUGT16tKmtrTV//ud/Hg4ojOflW7lypZk4ceIF2xjH2IurUzxdXV1qbGxUcXFxeJ/T6VRxcbEaGhpiWFl8OXr0qPx+f8Q4ejweFRUVhcexoaFBGRkZmjx5crhPcXGxnE6ndu3a1e812yIQCEiSMjMzJUmNjY06d+5cxFiOHTtW+fn5EWM5YcIEZWdnh/uUlJQoGAzq4MGD/Vi9HXp6erRhwwadOXNGPp+PMeyliooKlZaWRoybxO9ktD7++GPl5ubqxhtv1Lx589TS0iKJcbRBXH1Z4J/+9Cf19PRE/DJIUnZ2tj766KMYVRV//H6/JF1wHM+3+f1+ZWVlRbQnJycrMzMz3OdaEwqFtHTpUt12220aP368pC/GKTU1VRkZGRF9vzyWFxrr823Xiv3798vn8+ns2bNKT0/Xpk2bVFBQoKamJsYwShs2bNAf/vAH7dmz5ytt/E5evqKiIq1bt05jxoxRa2urVq1ape985zs6cOAA42iBuAooQCxVVFTowIEDeu+992JdSlwaM2aMmpqaFAgE9Jvf/Ebl5eWqr6+PdVlx59ixY/rRj36k2tpaDRgwINblxLVZs2aFfy4sLFRRUZFGjBihX//61xo4cGAMK4MUZ1fxDB06VElJSV9ZRd3W1iav1xujquLP+bG62Dh6vV61t7dHtHd3d+vkyZPX5FhXVlZq8+bNevfddzV8+PDwfq/Xq66uLnV0dET0//JYXmisz7ddK1JTUzVq1ChNmjRJNTU1mjhxop577jnGMEqNjY1qb2/Xn/3Znyk5OVnJycmqr6/X888/r+TkZGVnZzOevZSRkaFvfOMbOnz4ML+XFoirgJKamqpJkyaprq4uvC8UCqmurk4+ny+GlcWXkSNHyuv1RoxjMBjUrl27wuPo8/nU0dGhxsbGcJ9t27YpFAqpqKio32uOFWOMKisrtWnTJm3btk0jR46MaJ80aZJSUlIixrK5uVktLS0RY7l///6IwFdbWyu3262CgoL+ORALhUIhdXZ2MoZRmj59uvbv36+mpqbwNnnyZM2bNy/8M+PZO6dPn9aRI0eUk5PD76UNYr1KN1obNmwwLpfLrFu3zhw6dMgsXrzYZGRkRKyixhcr/Pfu3Wv27t1rJJl//Md/NHv37jX//d//bYz54jLjjIwM88Ybb5h9+/aZu+6664KXGX/zm980u3btMu+9954ZPXr0NXeZ8ZIlS4zH4zHbt2+PuBTxs88+C/d58MEHTX5+vtm2bZv54IMPjM/nMz6fL9x+/lLEGTNmmKamJrN161YzbNiwa+pSxEceecTU19ebo0ePmn379plHHnnEOBwO87vf/c4Ywxheqf99FY8xjOflevjhh8327dvN0aNHze9//3tTXFxshg4datrb240xjGOsxV1AMcaYX/ziFyY/P9+kpqaaW2+91ezcuTPWJVnn3XffNZK+spWXlxtjvrjU+Gc/+5nJzs42LpfLTJ8+3TQ3N0e8xqeffmruu+8+k56ebtxut/nhD39oTp06FYOjiZ0LjaEk89JLL4X7fP755+Zv/uZvzHXXXWfS0tLMD37wA9Pa2hrxOn/84x/NrFmzzMCBA83QoUPNww8/bM6dO9fPRxM7f/3Xf21GjBhhUlNTzbBhw8z06dPD4cQYxvBKfTmgMJ6X59577zU5OTkmNTXVXH/99ebee+81hw8fDrczjrHlMMaY2MzdAAAAXFhcrUEBAADXBgIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKzz/wF/Ga7vy6QObQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "env = gym.make('CartPole-v1', render_mode=\"rgb_array\")\n",
    "obs = env.reset()\n",
    "print(\"initial observation:\", obs)\n",
    "print(\"possible actions:\", env.action_space.n)\n",
    "\n",
    "\n",
    "def policy(observation):\n",
    "    return env.action_space.sample()\n",
    "\n",
    "\n",
    "for _ in range(5):\n",
    "    plt.imshow(env.render())\n",
    "\n",
    "    action = policy(obs)\n",
    "    print(\"\\ntaking action:\", action)\n",
    "\n",
    "    obs, reward, terminated, truncated, info = env.step(action)\n",
    "    print(\"got reward: {0}. New state/observation is: {1}\".format(reward, obs))\n",
    "\n",
    "\n",
    "# close the environment\n",
    "env.close()"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "include_colab_link": true,
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
